A Simple Practical Accelerated Method for Finite Sums
نویسنده
چکیده
We describe a novel optimization method for finite sums (such as empirical risk minimization problems) building on the recently introduced SAGA method. Our method achieves an accelerated convergence rate on strongly convex smooth problems. Our method has only one parameter (a step size), and is radically simpler than other accelerated methods for finite sums. Additionally it can be applied when the terms are non-smooth, yielding a method applicable in many areas where operator splitting methods would traditionally be applied.
منابع مشابه
Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex problems. We show that our method achieves a lower overall complexity than the re...
متن کاملA Few Finite Trigonometric Sums
Finite trigonometric sums occur in various branches of physics, mathematics, and their applications. These sums may contain various powers of one or more trigonometric functions. Sums with one trigonometric function are known; however, sums with products of trigonometric functions can become complicated, and may not have a simple expression in a number of cases. Some of these sums have interest...
متن کاملThe Almost Sure Convergence for Weighted Sums of Linear Negatively Dependent Random Variables
In this paper, we generalize a theorem of Shao [12] by assuming that is a sequence of linear negatively dependent random variables. Also, we extend some theorems of Chao [6] and Thrum [14]. It is shown by an elementary method that for linear negatively dependent identically random variables with finite -th absolute moment the weighted sums converge to zero as where and is an array of...
متن کاملOn dimensional regularization of sums
We discuss a systematic way to dimensionally regularize divergent sums arising in field theories with an arbitrary number of physical compact dimensions or finite temperature. The method preserves the same symmetries of the action as the conventional dimensional regularization and allows an easy separation of the regulated divergence from the finite term that depends on the compactification rad...
متن کاملFinite element simulation of two-point incremental forming of free-form parts
Two-point incremental forming method is considered a modern technique for manufacturing shell parts. The presence of bottom punch during the process makes this technique far more complex than its conventional counterpart i.e. single-point incremental forming method. Thus, the numerical simulation of this method is an essential task, which leads to the reduction of trial/error costs, predicts th...
متن کامل